In the current implementation of Multi-view Video Coding (MVC) compliant with the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 recommendation (hereinafter the “MPEG-4 AVC standard”), there is no provision to identify a specific view and to signal the camera parameters. This view information is needed for several reasons. View scalability, view random access, parallel processing, view generation, and view synthesis are all Multi-view Video Coding requirements which utilize the view information. Moreover, several of these requirements also utilize camera parameters which are currently not passed in a standardized way.
A method has been proposed to enable efficient random access in multi-view compressed bit streams. In the proposed method, a V-picture type and a View Dependency Supplemental Enhancement Information (SEI) message are defined. A feature required in the proposed V-picture type is that V-pictures shall have no temporal dependence on other pictures in the same camera and may only be predicted from pictures in other cameras at the same time. The proposed View Dependency Supplemental Enhancement Information message will describe exactly which views a V-picture, as well as the preceding and following sequences of pictures, may depend on. The following are the details of the proposed changes.
With respect to V-Picture syntax and semantics, a particular syntax table relating to the MPEG-4 AVC standard is extended to include a Network Abstraction Layer (NAL) unit type of 14 corresponding to a V-picture. Also, the V-picture type is defined to have the following semantics:
V-picture: A coded picture in which all slices reference only slices with the same temporal index (i.e., only slices in other views and not slices in the current view). When a V-picture would be output or displayed, it also causes the decoding process to mark all pictures from the same view which are not IDR-pictures or V-pictures and which precede the V-picture in output order to be marked as “unused for reference”. Each V-picture shall be associated with a View Dependency SEI message occurring in the same NAL.
With respect to the view dependency Supplemental Enhancement Information message syntax and semantics, a View Dependency Supplemental Enhancement Information message is defined with the following syntax:
view_dependency ( payloadSize ) {num_seq_reference_views ue(v)seq_reference_view_0 ue(v)seq_reference_view_1 ue(v)...seq_reference_view_N ue(v)num_pic_reference_views ue(v)pic_reference_view_0 ue(v)pic_reference_view_1 ue(v)...pic_reference_view_N ue(v)where num_seq_reference_views/num_pic_reference_views denotes the number of potential views that can be used as a reference for the current sequence/picture, and seq_reference_view_i/pic_reference_view_i denotes the view number for the ith reference view.
The picture associated with a View Dependency Supplemental Enhancement Information message shall only reference the specified views described by pic_reference_view_i. Similarly, all subsequent pictures in output order of that view until the next View Dependency Supplemental Enhancement Information message in that view shall only reference the specified views described by seq_reference_view_i.
A View Dependency Supplemental Enhancement Information message shall be associated with each Instantaneous Decoding Refresh (IDR) picture and V-picture.
At least one drawback of this method is the complexity introduced in the decoder due to the dependency being recursively obtained. Additionally, this method requires that every V-picture carry an SEI message (which is a non-normative part of the MPEG-4 AVC standard), resulting in the dependency being unable to be used for normative behavior such as reference picture selection.